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We develop dynamical programming methods for the purpose of optimal control of quantum states 
with convex constraints and concave cost and bequest functions of the quantum state. We consider 
both open loop and feedback control schemes, which correspond respectively to deterministic and 
stochastic Master Equation dynamics. For the quantum feedback control scheme with continuous 
non-demolition observations we exploit the separation theorem of filtering and control aspects for 
quantum stochastic dynamics to derive a generalized Hamilton- Jacobi-Bellman equation. If the 
control is restricted to only Hamiltonian terms this is equivalent to a Hamilton- Jacobi equation 
with an extra linear dissipative term. In this work, we consider, in particular, the case when control 
is restricted to only observation. A controlled qubit is considered as an example throughout the 
development of the formalism. Finally, we discuss optimum observation strategies to obtain a pure 
state from a mixed state of a quantum two-level system. 
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I. INTRODUCTION 

The dynamical theory of quantum nondemolition ob- 
servation, developed by Belavkin in the 80's [11, [3, Q re- 
sulted in a new class of quantum stochastic equations 
[Ij H H 01 for quantum dissipative systems under ob- 
servation. Different quantum jump and quantum dif- 
fusive stochastic equations are obtained when the ob- 
served quantity has discrete and continuous spectra as, 
e.g., in photon counting and homodyne detection of op- 
tical fields. In the last case, for linear models and initial 
quantum Gaussian states this allows explicit solutions 
in terms of a quantum analog of the Kalman linear filter 
first introduced in Ref.[l| (see also 3, i, [111,111 )■ 

These equations are similar to the classical filtering equa- 
tion derived by Stratonovich in the 60's [l^ for classical 
partially observed conditional Markov systems. 

In the classical theory of feedback control the so called 
Separation Theorem [l4 . [isl . [iGj applies. This theorem 
states that the full control problem can be reduced in 
two separated and independent parts: (i) the estimation 
of the state of the system; {ii) the optimal control of 
the system. The same approach can be applied to quan- 
tum systems, as first pointed out by Belavkin in Ref.[T7| 
and more recently implemented in the quantum dynam- 
ical programming method [1, [H, [ll]. The only differ- 
ence between classical and quantum systems is related 
to the concept of "state" given in the two theories. In 
both mechanics the state carries the necessary informa- 
tion to describe fully the system and this characterizes 



our knowledge of it. In classical mechanics a system is 
usually described by its position and momentum phase 
space variables. In quantum mechanics a state is de- 
scribed by a state vector which belongs to a linear Hilbert 
space or by a von Neumann density matrix. Contrary 
to the classical formulation, the linear property of the 
Hilbert space allows superpositions of quantum states. 
This, however, does not play a role in the separability of 
the state estimation and optimal control problems. The 
optimal control of the state of a quantum system implies 
the control of a density matrix valued stochastic process 
and is mathematically equivalent to any classical control 
problem. 

Experimentally, very important achievements have 
been obtained in the last decade which have led to ex- 
citing prospectives to manipulate quantum systems. For 
instance, high precision quantum measurements at the 
Heisenberg limit have been implemented [10, |2li and 
quantum feedback has been used to record an exter- 
nally provided quantum state of light onto an atomic 
ensemble [1^. At the same time several theoretical pro- 
posals f or q uantum state engineering like spin-squeezing 
[13, [2^ - | 26| . [ 2 7| . photon number states [13, entangled 
states |29|. l30l |3l| , quantum superposition states of opti- 
cal fields [34I and of two macroscopically distinguishable 
atomic states (33|, and cooling of either a mechanical 
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resonator |34| or the atomic motion in an optical cavity 
[35! via continuous measurement have been put forward. 
Among these activities more related mathematical issues 
as stability and observability become relevant and have 
been subject of investigation [ll,[l^, [36l. [37., ,38., ,39.,,4d|. 

We shall follow the approach of Ref . [l^ (which is also 
taken in most papers on quantum feedback control after 
Wiseman [llj), where the filtering controlled equation 
is postulated but not derived as a result of the condi- 
tioning of quantum dynamics upon the nondemolition 
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observations. To simplify the mathematics, wc shall con- 
sider only finite-dimensional models (like qubits). How- 
ever, unlike in [l9l |. we shall consider optimization of not 
only Hamiltonian control but also of control exercised 
by the choice among different observations carried out 
on the system. In the original setup [3, 01 , [l3| , of opti- 
mal quantum feedback control theory the cost and target 
functions were restricted to afhnc functionals of the state 
where these functions are therefore expectations of cer- 
tain observables. The result of minimization of an afRne 
function as expected cost of a controlled observable is not 
always affine but always concave. This point was argued 
in Ref. [i^l, and the use of nonlinear concave cost func- 
tions was justified by Jacobs [i^l, Wiseman and Ralph 
(43 | , Wiseman and Bouten for optimal feedback con- 
trol of qubit purification. We will thus consider concave 
cost and target functions of the properties of the system 
controlled by the feedback. 

The paper is organized as follows. In Sec. |TT1 we in- 
troduce the density matrix notation, time evolution gen- 
erators, and the notion of derivatives with respect to a 
quantum state. In Sec. Illli we introduce Bellman and 
Hamilton-Pontryagin optimality, and we introduce cost 
and bequest functions. In Sec. IIVI we turn to the prob- 
lem of quantum dynamics under observation and present 
the diffusive quantum filtering equation corresponding to 
homodyne or heterodyne measurements. In Sec. |Vl we 
analyze the optimal control problem with observations 
and constraints, and we derive a Bellman equation for 
filtered dynamics. In Sec. IVIl we discuss the special 
case of purification of a mixed quantum state by mea- 
surements. We conclude the paper with a discussion in 
Sec. IVlIl 



II. STATES, GENERATORS AND 
DERIVATIVES 



We will assume a complex, finite dimensional, Hilbert 
space I) = for our open (observed and controlled) 
quantum system. It is convenient to define the quantum 
state space S as the compact, convex set of positive and 
hermitian density matrices, normalized with respect to 
the unit trace, trjgi} = ^Tr{g} = 1. 

The real linear combinations of such matrices form the 
linear space C of Hermitian matrices. In general every 
state g can be parametrized as g{q) = Qo^q with respect 
to a given state qq € S hy a, corresponding q € Co, where 
Co <Z C is the space of trace zero matrices. 

Example: Throughout this text, we shall illustrate 
and apply our results and formalism to the example of 
a single two-level quantum system, also described as a 
qubit. Any qubit density matrix with respect to the nor- 
malized trace tr{-} := iTr{-} can be expanded on the 
Identity and the three Pauli matrices, g = I + ap, with 



o'r = i^xO'x + i^y<^y + i^z^z = r ■ . Thc quantum state 
space S thus becomes associated with the unit ball (Bloch 
sphere) {r : \r\ < 1}. 

For the qubit we have q = —ap with respect to the 
central go = I- Thus q (g) can be identified with the 
Euclidean vector q = —7^ globally parametrizing p G 5 as 
g = I - cTg-. 

The quantum state Master Equation (ME) describing 
an open quantum system is defined as 



^^g' +v{u{t),g')={), 



(1) 



where g* ^ S and the generator or drift term v is given 
by 



V {u, g) = ^[H (u) ,g]+Y^ vl, («) {g) , (2) 

3 

VL,{g) = 2 L^gV\ (3) 

with H (u) = H (m)^ self adjoint, and U (u) belongs to 
the complex space S ([)) of bounded operators on [) for 
each value of the set of control parameters u. The param- 
eter u = u{t) is the admissible control trajectory which 
may be restricted to a domain U{t) C M" of dimension- 
ality n, possibly depending on time t. 

Example: Assume that the qubit dissipative dynam- 
ics is given by a Hamiltonian H (u) = -Ictu controlled by 
the magnetic field u {t) e R'' for d < 3, and by a single 
dissipation operator L = \\(Jz. With the Pauli matrix 
representation g = I — for the density matrix, the ME 
evolution is governed by 

v{g,\i) = a ■ q) — {q^^a^ + qyCTy) . (4) 

A (nonlinear) functional i-^ F [gi] admits a derivative 
if there exists a B (f))-valued function VgF [•] such that 



hm -{F[. 



H-F[-]} = (r,V,F[.]) Vre£o, (5) 



where we have introduced the pairing {g,X) := ti {gX} 
with g £ S and X E C* , where C* ~ B (()) is the adjoint 
space of C. 

If gi* is a quantum state trajectory controlled by equa- 
tion ([1]), then we may apply the chain rule 



^ For vectors in M? we use the arrow notation, e.g., r. To avoid 
misunderstandings and to simplify the notation we use bold Latin 
characters to indicate the projection r of a three dimensional 
vector ron a subspace M"^ C with ci < 3. 
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for such a functional F. 

A Hessian V®^ = (g) Vg is defined as 

lim i (r', V,F [q + hr] - V,F [g]) = {r ® t' , Vf F [g]) , 

with r, r' G £o a-nd we say that the functional is twice 
continuously differentiable whenever V®^F [q\ exists. 

Example: Let F [p] = / be a smooth function of the 
state, i.e., of q. Then VgF[£i] can be directly identified 
with — (?• V/ [q) in the sense that (r, VgF [g]) = —t-Vf (q) 
for any r = cTj- e £o- Here the minus sign is related to the 
fact that the state g is identified with r, but the gradient 

f (q) is considered with respect to q = — r. 

Similarly we can write 

VfF[l-a,^=(a.v) .f{q). 

III. BELLMAN AND 
HAMILTON-PONTRYAGIN OPTIMALITY 

A. Cost functions 

Let us consider the integral cost for a control function 
{u (t)} of the quantum state g* over a time-interval (ig, T] 

J [{u (t)} ; to, go] = / C {u (t) , e*) dt + G{u (T) , g^) , 

■/to 

(6) 

where {g* : t € (tQ,T]} is the solution to a quantum con- 
trolled ME with initial condition g*° = go, and C is a cost 
density while G is the terminal cost, or bequest function. 
Causality implies that for any t £ [0, T] the state g* de- 
pends only on u{t') with t' e [0,t) and is independent of 
the current and future values of u{t). In particular, the 
choice of u(T) at the terminal time instant T does not 
affect the state g^. We emphasize that the admissible 
control strategies u{t) are not necessarily continuous but 
they can be assumed right continuous for all t with left 
limits u{t-) not necessarily equal to u{t). One can thus, 
for any t, regard u(t) as entirely separate from earlier val- 
ues u{t' < t), while any later value u{t" > t), serves as a 
"postprocessing" control for g*. Our task is to adapt u{t) 
to minimize the contribution from the cost density and 
to use u{T) to "postprocess" the final quantum state or 
to modify the terminal cost or bequest function in Eq. ([6]) 
to most successfully achieve the desired goal. This will 
be exemplified in the following. 

B. Quantum dynamical programming 

Let us first consider the quantum optimal control the- 
ory without observation, assuming that the state g* € S 
obeys the ME ([1]). To identify the optimal control strat- 
egy {u (t)} with the specific cost J [{u} ;to, go], we note 



that for times t < t + h < T, one has 
Sit,g) :=inf | r\{u{r) , g^)Ar 

{«} yjt 

+ r C{u{r),g^)dr + G{uiT),g^)\. 

Jt+h J 

Now, we assume that {u° (r) : r e {t, T]} is the optimal 
control when starting in state g at time t, and denote by 
{g^ : r € it,T]} the corresponding state trajectory g^ = 
g^{t,g). According to Bellman's optimality principle^ 
the control {u° {r) : r G {t + h, T]} is then optimal for the 
evolution starting from at the later time t + h, and 
hence 

S {t, - inf 1^ ^ Ciu (r) , g^) dr + S{t + h, g'+'') | . 

For h small we expand — g — v {u {t) , g) h + o (h) 
and we may apply a Taylor expansion of S {t, g). Then, 
by taking the limit /i ^ we obtain [Toj 

- {t, g) = inf {C [u, g) - {v {u, g) , V,S {t, g))] . (7) 

This equation should be solved subject to the terminal 
condition 

S{T,g)^ini{Q{u,g):u(,U{T)} = ST[g]. (8) 

u 

We recall that when the infimum is reached, then the 
objective of the optimization problem we aim to solve is 
obtained. This formalism has been applied in the case of 
optimal control of the cooling of a quantum dissipative 
three- level A system and in the classical thermody- 
namic optimization of the evolutionary Carnot problem 

C. Quantum Pontryagin Hamiltonian 

We introduce the Pontryagin Hamiltonian function de- 
fined, for q £ Co, p G C* , by 

H„(q,p) -.^ sup {{v{u,g{q)) ,p) -C{u,g{q))}. (9) 

We use the parametrization g (q) ~ go~ qhy a, zero trace 
operator q G Co and v is the velocity q of q = go — g. 
The equations of motion for the state operator q and 
for the operator p can be expressed with the Pontryagin 



^ The optimality principle states that if the path of a process 
through the stages ia ^ ifc — > tc is the optimal path from ta 
to tc, then the path from tf, to tc is optimal as well [Tsl. |46|. . 
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Hamiltonian in formally the same way as the equations of 
motion for the canonical coordinates {q,p) in the Hamil- 
tonian formulation of classical mechanics [l^ . 

Since {v (w, g) ,1) = 0, the Pontryagin Hamiltonian 
does not change if we replace any p e by p + A/ with 
A G C. The mathematical consequences of this equiva- 
lence class property and the observation that the opera- 
tor p in Eq. ^ is the Legendre-Fenchel transform of the 
cost function C are further developed in Ref . [l9| . 

We may use the Pontryagin Hamiltonian to rewrite 
([7]) as the (backward) Hamilton- Jacobi-Bellman (HJB) 
equation 



d_ 
dt 



S (i, Q (g)) + (V,S (t, g))) = 0, (10) 



which can be simply written as dtS (t, g) = 
Hf (fi'o ^ (VgS (t, £>))) for any a priori chosen 
reference state go G S. 

Example: In the case of the Hamiltonian controlled 
dissipative dynamics (jl]) we have with p = + CI 



{v,p) ^{cfxpj-u- 



(qxPx + qyPy) 



For the qubit with the density cost 



C(u,e) = 0^,(u), 0+ (S) = 



0, u = ue Bi 

+00, u 4 El 



under the constraint Bi = {u eU : |u| < 1}, the supre- 
mum in Eq.Q, 



^viq,p) ■■= sup {{v{u,g{q)) ,p)} 

|u|<l 

iap 

= sup {u- {qxp}} — {q^p^ + q.yPy) , 

u|<l ^ 

is achieved at the stationary point u° (q) = 
P{t,q) /\pit,q)\, with p = {qxp)^ denoting the 
projection of q x p onto U, where the costate 
p (i, q) = —VqS (t, q) is obtained from the solution 
slt,q) = S{g{q)] of the HJB equation This yields 

the Pontryagin Hamiltonian with dissipation 



H. iq,p) = |P(M-)I + ^ {qx^^ + qy^J s [t,q) . 



D. Linear, affine, and concave cost and bequest 
functions 



Now, we consider cost density and bequest functions 
which are linear functions of the quantum state g 



i.e., they can be interpreted as the expectation values 
of a cost observable C (u) and a bequest observable G (u), 
which may depend on the control parameter u G M". One 
can consider, for example, an average energy associated 
with the control parameter as a cost, say C (m) = 
and the error probability G = {g,I — Pt) given by the or- 
thoprojector Pt = |'0t)(''/'t| on a target state- vector \iI)t) 
as the bequest observable. Although the dependence of 
G on the final value u (T) of the control parameter in 
Eq.® is sometimes redundant, for the sake of generality 
and for reasons which will be clear below we keep this 
dependence. 

It is natural to extend the cost and bequest functions 
to affine functions 



C(u, £») = {g,C{u)) +c(m), 
G(u, £») = {g,G{u)) +g{u), 



(12) 



of the state g which can be obtained from the linear func- 
tions by replacing C i-^ C + cl and G i-^ G + gl in 
pip . This generalization has no consequences, unless the 
real- valued functions c (u) and g (u) are allowed to take 
also the infinite cost value -|-oo, thus reflecting rigid con- 
straints on u. Indeed, any constraint on the admissible 
domain U (t) , can be described by the infinite costs 

c (t, u) = oo Vu(^U {t) , g{u)^(x, ^u^U (T) , 

such that the expected cost is finite only if u (t) is in the 
allowed domain U (t). For instance, in the deterministic 
preparation of atomic Dicke states of Ref. [lOl the con- 
trol u is the strength of a magnetic field, which in an 
experiment cannot assume arbitrarily large values. 

Of course there is a range of useful cost and bequest 
functions which cannot be cast into an affine form. For 
example, the variance of a certain observable, the von 
Neumann entropy of the state of a quantum system, or 
sub-system, the purity of a quantum system character- 
ized by the trace of the square of the density matrix, 
and various entanglement measures are important quan- 
tities used to characterize desirable properties of quan- 
tum systems, e.g., in precision metrology and quantum 
information theory. 

As described above, if we aim to produce a definite 
pure target state \iPt), we will maximize the expecta- 
tion value of the particular pure state projector Pt = 
|V't)(V't|, and we thus have a linear bequest function. If, 
however, we only wish to maximize the purity, but we do 
not care precisely which state is produced, for a given g, 
we could look for the nearest pure state, and maximize 
the expectation value of the corresponding projection op- 
erator, and since that projector now depends on g, we 
effectively obtain a non-linear bequest function. 

The search for "the nearest pure state" can be 
parametrized by the "post-processing" u(T) dependence 
of a linear bequest observable, and we can more gener- 
ally write the minimization of quantum state controlled 
functionals of the type ^ as 



C {u, g) = {g, C (u)) , G (u, g) = {g, G (u)) , (11) 



S [g] = inf {(£-, G (u)) : u G U{T)} . 
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Example: Consider the affinc bequest function with 

G {u,g) = O^^ (u) - q- u, 

of the qubit state g ~ l — (Jq corresponding to the general- 
ized qubit cost observable G (u) = O^^ (u) I + as, includ- 
ing the constraint function (u) for the unit ball Bi = 
{u e K'^ : |u| < 1} in d < 3. Then S [g] = Ms {g, G (u)) 
is the closed concave function 

S [g] = inf { {g, as) + Ot (u)} = - sup (f • u = - |q| , 
um^ ueBi 

(13) 

where q is the projection of (f e R'^ onto R'^. In this way 
we recover the concave bequest function S (T, g) = ~ |q| 
used as a measure of purity by Wiseman and Boutcn for 
d = 2 in Ref. 



IV. QUANTUM DYNAMICS UNDER 
OBSERVATION 

A. Quantum measurements and posterior states 

The state of an individual continuously measured 
quantum system does not coincide with the solution of 
the deterministic ME ([1]). but instead depends on the 
random measurement output in a causal manner. The 
posterior density matrix should be viewed as an 5- 
valued stochastic process gl : w i-^ g^, causally depend- 
ing on the particular observations = {y„ (r) : r < t}, 
which are, in turn, obtained with a probability dis- 
tribution determined by the previous posterior states 
{g, '■ r < t}. Here the symbol • denotes a random vari- 
able, when its actual value to is not displayed. 

The causal dependence of the posterior state £>* on 
the measurement data is given by a corresponding 
quantum filtering equation derived in the general form 
by Belavkin in [a, [3|- The quantum trajectories, in- 
troduced by Carmichael [5l| , and the Monte Carlo Wave 
Functions (MCWF), introduced by Dalibard, Castin and 
M0lmer [s^, \E^, are stochastic pure state descriptions 
of dissipative quantum systems. In these approaches the 
dissipative coupling to a reservoir and resulting mixed 
state dynamics of a small quantum system is "unravelled" 
by simulated Gedankenmeasurements on the reservoir. 
These descriptions are included in Belavkin's formula- 
tion, which, however, does not assume a complete detec- 
tion of all reservoir degrees of freedom, and hence it re- 
tains the density matrix description. More importantly, 
however, it does not only deal with the simulation of the 
unavoidable dissipation of a quantum system, but also 
with the dynamics induced by the probing of the system 
by coupling to a measurement apparatus. One may, for 
example, probe atomic internal state populations and co- 
herences in a single atom, or a collection of atoms, by the 
phase shift or rotation of field polarization experienced 



by a laser beam interacting with the atoms. This mea- 
surement may be turned on and off, and several measure- 
ments may go on simultaneously as controlled by the field 
strengths of different probing laser beams. Here for sim- 
plicity we display only the diffusive case corresponding 
to homodyne or heterodyne detection in optics. These 
detection schemes were identified as continuous limits of 
the Monte Carlo Wave Function quantum jump dynam- 
ics, associated with photon counting experiments with 
strong local oscillator fields [13, [sHl ■ 

The quantum diffusive filtering equation as derived in 
[3, @1 for probing by coupling to a single set of system 
observables L, has the form 

df?l + V {g\) dt^e {g\) dw (t) , (14) 

where the time coefficient v contains the commutator 
with the Hamiltonian and the damping terms in the de- 
terministic ME ([T|). The right hand side of the equation, 
where dw(t) denotes an infinitesimal standard Wiener 
Gaussian process with dw'^{t) = dt, provides the fluctua- 
tion innovation term, 

dw{t) = dy,{t)- {gl,L + L'i)dt, (15) 

governed by the difference between the random outcome 
of the measurements and its expectation value. This term 
acts on the density operator as specified by 

9{g) = Lg+gL^ -{g,L + L^)g. (16) 

In optical homodyne detection the term dyui {t) in 
Eg. p^ describes the continuous photocurrent, which is 
the output signal obtained from the detector. 

Hereafter we shall use negative integers j_ to indicate 
the damping dissipative operators and positive in- 
tegers j+ to describe the dissipative operators U+ due 
to the coupling of the system with the measurement ap- 
paratus. The same notation will be applied to the drift 
term v{g) = "j- {q) + wo(e') + ^]+ io)- where 

vo{g)^'-[H,g], (17) 

.,^ig)^ ^''^^''^^X^^''^^'''^ -V^giL^^)^. (18) 

Example: Assume an undamped qubit system with 
vanishing Hamiltonian, and consider the probing de- 
scribed by the observable = = L with A S R. Us- 
ing the Bloch vector notation for the system density ma- 
trix we can write Lg+gL'' = A (cr^ 4- z) and (^g, L + L^^ = 
Az, where we used the fact that q = — {x, y, z) = — r. 
Therefore, the drift term vi and the ffuctuation coeffi- 
cient 9 in the filtering equation (|14|) are given by: 

A^ 
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{g) = A (1 - z^) - zap± 



A[(l 



where ri = f — {r ■ n)n with n being a vector in M.^ of 
unit norm. In our specific case n = Cz = (0,0,1), and 
rg = (x,y,0). The fluctuation coefficient can also be 

rewritten as 9 {g) — a^, where I = A {—xz, —yz, 1 — z^). 
The innovation process driving the qubit filtering equa- 
tion is defined by dy^ (t) — X {g^j, az) dt. 



B. Average change of stochastic functionals 

Let {e^ {t, g) : iv € fl} he the solution of (Il4|) for r > t 
starting in state = p at time r = t for all uj & fl. Then, 
for a smooth functional F on £, we have the average rate 
of change 



lim i 1 1 

h\0 h 



[F [g'^"^ it,g)]^F[g]\g]}=Dit,g)F[g] 



where Ei'lfi*] denotes the average of a functional of the 
stochastic state g at time t. Since the change in g con- 
tains both deterministic terms, linear in dt, and fluctu- 
ating terms, scaling with y/dt, we apply the Ito rule [s^ 
and expand the function to second order in small varia- 
tions to get the correct average rate of change. Hence, 
the elliptic operator D {t, g), in the diffusive case is 



D it, g) F [g] = 
where the Ito correction is given by 



"(t,e),V,F[e]) + -A,F(i,e), (19) 



A,F{t,g) = (0it,gf\vf'F[g] 



(20) 



For an N level system, whose state can be described by 
a generalized Bloch vector r in the notation V®^ 

reads as Vf ^ = (Vr)Vr , where Vr is the N'^ — 1 column 
gradient vector operator and (Vr)^ is its transpose. The 
same notation applies for the operator 8 (t, g)^ . 

Example: Let us illustrate the above expression for a 
functional F [g] = f{r), where r = |r| is the length of the 
Bloch vector. 

We consider again the situation of the previous exam- 
ple, where = ^(Tz = L. Then, we obtain 



(vi (g) y ,F [g]) 



A2 



V/(r) 



where we used the result of the third example in Sec. |TT] 
for VgF [g]. Then, the operator 9 (g)'^'^ can be written as 
the matrix 



i®i = y 




—zx 
-zy 
l-z2 



with / given in the previous example, and V^^F[£i] can 
be identified with the Hessian matrix Sj [/] (r) as discussed 
in the third example of Sec. HH The Ito correction ([20|l 
is hence given by 



AgF(^,g) 
A2 



where f^y 




fxx Ixy fxz 
fyx fyy fyz 
fzx fzy fzz 




dxdy ' 



Since / depends only on r and V f{r) 
Hessian matrix ^ [f] (r) can rewritten as 



'^rdrf, the 



19/ 
r dr 



1 fd'f lar 



X xy xz 
yx y^ yz 
zx zy z^ 



Hence, we have 

\l\'df X' fd^f Idf 
r or \ or^ r or 

and the elliptic operator in becomes 



z\l-r^)\ 



D [t, g) F [g] 




2^2 \ Qj.2 



z'[\-r'Y 



For instance, if /(r) = 1 — r^, then the second line in the 
above equation disappears and therefore D [t, g) F [g] — 
A2(r2-l)(l-z2). 

Simultaneous probing of different observablcs, repre- 
sented by operators , coupled for example to differ- 
ent probing light beams, leads to a vector of random 
measurement outputs, and is governed by the filtering 
equation 



dgi + V (gi) dt^J2 W (21) 



and with 



9^+ (g) = L3+g+ g{P+)'^ - (g, -f {P+)'^)g. (22) 

Note that both the measurement induced terms and 
the diffuse operation term contain the relevant probing 
strengths through the magnitude of the operators . 
In the following section we shall treat these measurement 
strengths as our control parameters, and see how a sys- 
tem is optimally controlled by measurements alone. 

Example: In the above example we showed the ex- 
plicit case of probing cr^ with a coupling strength A. By 
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cyclic permutation of the coordinates {x,y,z) we obtain 
the equivalent expressions for probing along the other 
coordinate axes, and by continuous rotation of {x, y, z) 
the effect of probing along an arbitrary direction can be 
derived. 

Assuming L" = ^cFji with n of unit norm, wc can easily 
generalise the previous results: 



6{q) ~ fjf with I = \ [n — {n ■ r)r], and 



V. OPTIMAL FEEDBACK CONTROL WITH 
CONSTRAINTS 

We assume a quantum system under continuous obser- 
vation described by the filtering equation (|14p. A choice 
of the control function {u (r) : r G [to,t)} is required be- 
fore we can solve the filtering equation at the time 
t for a given initial state go at time to. To this end, we 
define the optimal average cost on the interval [to,T] to 
be 

S{t,g) := inf E [J. [{u. (t)} ; to, p]] , (24) 
{"•} 



df A2 fd^f Idf 



dr 



dr 



2\2 



(n-r)^(l-r') 



Hence, the elliptic operator (jTH]) reads 



D{t, ^^)F[^?] = — (1 



where |[[^ has been written explicitly. Since D (t, g) F [g] 
provides the average change of F [g] for the system while 
monitoring L", we are now able, for any g^ to make the 
optimum choice of observable Lfj which, locally in time, 
gives the largest change. We want F [g] to decrease as 
fast as possible, and hence 



Qj.2 



Idf 

r dr 



df 
or 



(23) 



must be satisfied. If we assume that df.f < 0, then the 
coefficient in the square brackets in (j23p is positive when 



Qj,2 



< 



df 



r{l 



dr ' 



which implies that drf must be negative, i.e., /(r) is 
monotonic. Then, the minimum is reached for (n-f) = 0. 
When we consider the function /(r) = 1 — r^, the above 
conditions are satisfied and it tells us that it is optimal 
to perform a measurement in the orthogonal direction n 
with respect to the state r, when our objective is the pu- 
rification of the state. This conclusion was also obtained 
by Jacobs in Ref.ji^. 

Note that in the example we showed that a proper 
choice among measurements leads to the highest local 
increase of the purity, but we remind that this may not 
necessarily be the optimal way to obtain high purity of 
the final quantum state after a total probing time T. 
In the following section we shall treat the measurement 
strengths as our control parameters, and apply the Bell- 
man principle to identify how a system is optimally con- 
trolled by measurements alone. 



where the minimum is considered over all admissible mea- 
surable control strategies {u, (t) : t > to} adapted with 
respect to the innovation process in Eg. psp . By ad- 
missible we mean any stochastic process Uuj{t) for which 
the controlled filtering equation is well defined and has a 
unique solution glj (for more precise mathematical defini- 
tions see, for example, [s^). The aim of feedback control 
theory is then to find an optimal control strategy {u° (t)} 
and evaluate S (t, g) on a fixed time interval [to, T]. 



A. Optimality equation for observed systems 

We consider the problem of computing the minimum 
average cost in p4|. Even though the cost is random, 
the Bellman principle can be applied also in this case. 
As before, we let {u° (t)} be a stochastic control leading 
to optimality and let p° (r) be the corresponding state 
trajectory (now a stochastic process) starting from g at 
time t. Again choosing t < t + h < T, we have by the 
Bellman principle 

E [S (t -I- h, g° {t + h))] + o {h) = S (t, g) 
+ inf I ^ (t, g) + C (li, g) + D (u, g) S (t, e)| h. 

Taking the limit h yields the quantum backward 
Bellman equation 



dS 



[g]^m{{Ciu,g) + Diu,g)S[g]} (25) 



as derived in |4l].[57l|. 

This can be rewritten in the generalized HJB form as 



d_ 

dt 



S(t,e(g)) = Hf,(g,VfS(t,e(g))) (26) 



in terms of the generalized (Bellman) " Hamiltonian" 
which takes in the diffusive case the second order deriva- 
tive form 

K^, (g, V®S) ~ sup {(,; {u, g {q)) , V,S) ~C{u,g {q)) 



--Ag5{u,g) 
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where A^S (w, g) is defined in pO]) . This equation is to be 
solved backwards with the terminal condition S (T, g) = 
ST{g). 

If 9 in p4p and (fT6|) docs not depend on u (the control 
is only in H and not in L-'^ , L^+), this gives the diffusive 
HJB equation with a possible nonlinear dependence only 
on the first derivative VgS [g]: 



dS 

"at 



+ H, (q,VgS) 



02 



Exactly as in the deterministic case, the solution S of this 
diffusive equation defines the optimal strategy through 
V,S(t,e). 

In the case we control the strength of different kinds 
of measurements carried out on the system, 9^+ and the 
associated drift term components depend on the control 
parameters u. Then, the optimality equation is nonlinear 
only in the Hessian 



dS 
'dt 



K(e),V,S) + H^ 



(g,V,S,VfS) 



(27) 



where vq is defined in p7|) . and Hg is defined later in 
Eg. ((28)) . Here we assume the possibility that we can con- 
trol also the dissipative channels, and therefore their drift 
terms Vj_ (g) arc included in the Hamiltonian Hg. 



VI. OPTIMAL CONTROL OF PURIFICATION 

A. Generalized Bellman Hamiltonian 

A pure Hamiltonian control does not change the eigen- 
values of quantum states and therefore does not change 
the entropy of any state as a natural bequest function 
of the purification. On the other hand, the filtering dy- 
namics ([Til) changes the entropy, as it provides the state 
conditioned on measurements. Thus, both a Hamiltonian 
feedback together with the continued probing of the sys- 
tem, and a feedback strategy where new measurements 
are selectively carried out on the system, can be applied 
to optimize the convergence towards a pure state. The 
former possibility has been studied by Jacobs [i^ , Wise- 
man and Bouten [i^. Instead, we shall assume control 
of the coupling A to the continuous measurement in the 
filtering equation as a real function of time t and the in- 
formation previously obtained. Thus, we are interested in 
the optimal purification strategy via the feedback mea- 
surement control in one or several channels by solving 
the optimality equation ((27|) with the Hamiltonian 
containing explicitly all the diffusive measurement terms 



and the corresponding dissipation drifts v 



We 

allow, however, also the control of the dissipative oper- 
ators which correspond to some unobserved modes with 
velocity term Vj_ . 

We may assume that the operators — W , with j 
being either or j_ , are controlled only by the strengths 



Uj = I I or by the phases Uj = arg A^ of the cou- 
pling parameters A^ with fixed measurement operators 
W . Taking the first (controlled strength) choice with 
arg A-' = 0, we define the corresponding Vj — UjVj^j, 



9^ 



and A^^S 



rescaled Vjii and Ojij . We then obtain 



, Vf^S) in terms of the 



Hg := sup 

{uj<0} 



C(u) 



(28) 

explicitly in terms of the measurement strengths Uj. 

Thus, we have a convex optimization problem under 
the constraint u-' > 0. We will consider the optimiza- 
tion problem under the further natural constraint of a 
given maximum total probing strength, ||u|jj^ :— '^jU^ < 
1, which is incorporated by choosing the cost function 
C (u, g) = +00 if ||uj|j^ > 1, and C (u, g) — otherwise. 
Equation ([^5]) reduces to 



max <^ {vj (g) , VgS (t, g)) 



Al5it,g)} (29) 



if at least one value under the maximum is positive, and 
u° (t) = 1 for any optimal j — {g,S) and u° (t) = 

7^ jo- Otherwise, Hg = 0, no measurement purifies 
£>, and the maximum is achieved on the optimal feed- 
back strategy u° (t) = 0. In the case of a single chan- 
nel measurement this defines a simple two- valued strat- 
egy for when the probing should be switched on and off, 
M € {0, 1}, corresponding to the Hamiltonian 



H 



1 



{v^ig),VsSit,g))^ -AlS {t,g) 
where \x\, = max{0,x}. 



(30) 



B. Purifying a qubit only with measurements 

Let us take the cost c (u) ~ (u) of the constraint 

U = |it-' > : J2j < and the hermitian operators 

j^nu _ hiBlfj^^^^ where instead of the integer j+ we use 
the outward normal unit vector, n^, parametrized by 
the continuous solid angle argument, f2, along which the 
diffusive measurement is performed. In the following we 
set A(f2) = y''u(fl) — 1, as discussed in the previous 
section. Besides, we assume that the system is subject 
to no dissipation and no Hamiltonian control. In this 
case, the diffusive filtering equation (|14p reduces to 



dgi + vn (gl) dt = 0" (gl) dw (t) 



(31) 



This equation can be rewritten in terms of the state vec- 
tor r as 
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dx\ I x~ [n- r)nx \ f\t / - (n ■ r)x \ 
dy + {n- r)ny — = Uy ~ [n ■ r)y dw, 

dz / \ z — {n ■ r)nz j \ — (n • r)z I 

where we used the results of the examples in Sec. IIVI 
As noticed in the last example of Sec. IIVI about the 
purification of a qubit state, the elliptic operator 



attains its largest negative value if f? • r = 0, that is we 
should observe the system in a Blocli vector direction 
orthogonal to the one of the state density matrix. This 
simplifies the diffusive filtering equation considerably. 

In our example, the HJB equation for the optimal con- 
trol of qubit purification (in its very compact form) is 
given by: 



dS 
'di 

1 , 



V ri • VS • fi - fi • VS 



where (5(nf2 — n^-i^) is the Dirac delta function on the 
surface of the unit sphere, uq, = ■ f, and S,-jjj — 

If an = is chosen, then ri^ = r — aji^fj = ^ a-nd the 
above equation simplifies to 



dS 1^ 



0. 



If we write r = r(sin (^sin??, cos sini?, cos?9) this equa- 
tion can be reduced to 

It is straightforward to check that S{t, r) = 1 — r^e"'-^"*' 
solves dsn with S(r, r) = 1 - r^. Note that this solution 
is obtained under the assumption that we always measure 
a Bloch-sphere component orthogonal to the current den- 
sity matrix Bloch vector. It is, however, easy to verify 
that the supremum according to (j29p is in accord with 
that choice. This confirms the demonstration by Wise- 
man and Bouten in Ref. [1^ of the optimality of Jacobs 
[ist purification protocol. 



C. Controlling only the fluctuations 9-'+ 

As a special constraint on our control consider u = 
(uj ) indexed by j± ~ ±j and add the constraints Uj_ = 
1 — u.i , with R^- = for all j — 1, . . . ,n under the 



constraint ^ 



j>i "J 



< 1 (and 



n). The new 



constraint corresponds to keeping the dissipation drifts 
for each pair independent of the controls, V-j + 

u-i-j = vj^j and the optimality equation reduces to 



dS 



{v {q) 



V,S) 



iminA^,S(i,e). (33) 



Hence, — {v (g) ,V gS) + H, where the new Hamil- 
tonian, H, is defined by the minimal Hessian A^S (t, g). 



Equation ([351) becomes linear if one of the Hessians, say 
A],S {t, g), is the most negative, A^S < A|,S for all j, g 
and t. 

The constraint Uj_ = 1 — Uj^ with R^^ = R^+ cor- 
responds, for example, to the partial monitoring of a 
dissipativc channel which leaks information to the en- 
viroment. Such a leakage can be, for example, the one 
of resonance fluorescence from an atomic quantum sys- 
tem monitored with finite detection efficiency or within 
a finite solid angle, proportional to v+j. 

The qubit purification protocol we have discussed in 
Sec. IVI BI does not contain dissipation, as in Refs.[43l.[45|. 
Those results would be changed in the presence of pure 
damping terms (j < 0) and it would be more difficult 
to obtain a pure state. Here the choice of constraint 
Uj_ = 1 — Uj^ allows us to reduce the diffusive filtering 
equation of the observed system to a stochastic master 
equation where no dissipative terms appear. More pre- 
cisely, the equation would have only the drift terms v . 
For instance, the dynamics of a damped qubit, with only 
a dissipative term v-i and a measurement observables 
L^, as before, would be governed by a filtering equation 
formally identical to (j3ip . but now with the possibility 
to control only the fluctuation operator 9^+ . Again, how- 
ever, we can decide to perform a measurement or not by 
looking at the minimal Hessian A^S (t, g). 



VII. DISCUSSION 

We have presented a general formalism for the optimal 
control of a quantum system subject to measurements, 
where the control can both be of a suitable feedback 
Hamiltonian and of the choice of future measurements 
carried out on the system. The use of measurements to 
prepare and protect pure and entangled quantum states 
can thus be made subject of systematic investigation, and 
optimal schemes can be dc viced for given physical setups. 
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It is the philosophy of our work that the quantum state 
of a controlled system, i.e., its von Neumann density ma- 
trix, is treated in the same way as classical control engi- 
neers treat the state of their classical systems. The Bell- 
man principle can then be applied in in the same way 
as for classical states. In the present work we derived 
the corresponding Hamilton- Jacobi-BcUman theory for a 
wider class of controls and cost functionals than tradi- 
tionally considered in the literature. 

Another interesting problem, which is explicitly solv- 
able but is formulated in the infinite dimensional Hilbert 
space, is the setup problem for the quantum feedback 
control with "soft" constraints given by quadratic cost 
functions c and g in m [1, 0], [i3|j It reduces 

to the linear optimal control problem in the finite- 
dimensional space for the sufficient coordinates of the 



Gaussian Bosonic states exactly as in the classical linear- 
quadratic Gaussian case. For a more detailed discussion 
with proofs we refer to 3, and for a particular case in a 
more recent work by Yanagisawa [39| . 
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